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THE  RELATIONSHIP  BETWEEN  THE  MOVING  TARGET 
RIFLE  MARKSMANSHIP  TRAINER  (MTRMT) 

AND  THE  MULTIPURPOSE  ARCADE  COMBAT  SIMULATOR  (MACS) 


INTRODUCTION 


Given  the  recent  increase  in  the  cost  of  live  ammunition  and  decrease  in 
the  cost  of  microcomputers,  the  Army  is  becoming  increasingly  interested  in 
weapon  simulators.  In  addition  to  the  potential  cost  savings,  simulators  may 
actually  contribute  to  effective  training  in  ways  that  are  difficult  to  achieve 
in  actual  live  fire.  For  example,  immediate  and  precise  feedback  about  hit  or 
miss  shot  location  are  standard  features  in  a  simulator  while  often  difficult  to 
provide  on  a  live-fire  range.  The  purpose  of  the  present  study  was  to  measure 
the  relationship  between  two  microcomputer-based  marksmanship  simulators. 

One  of  the  simulators  was  the  Moving  Target  Rifle  Marksmanship  Trainer 
(MTRMT),  a  prototype  marksmanship  trainer  developed  by  Spartanics  Ltd.,  Rolling 
Meadows,  Illinois.  The  MTRMT  has  three  major  components:  the  rifle,  the  target 
assembly,  and  the  console  (see  Figure  1).  The  rifle  is  a  simulated  M16A1 
standard  service  rifle.  The  weapon's  adjustable  recoil  is  produced  by  a  metal 
rod  that  is  attached  to  the  barrel  of  the  weapon.  The  sound  of  the  report  is 
also  adjustable  and  delivered  through  headphones. 

The  target  assembly  can  display  any  of  a  number  of  scaled  silhouette 
targets  representing  a  number  of  distances:  50  m,  100  m,  150  m,  and  250  m. 

Each  "E-type"  target  represents  the  head  and  torso  of  a  man.  The  user  can 
select  targets  which  are  stationary  or  moving.  If  moving  targets  are  selected, 
then  the  user  can  further  decide  if  the  targets  move  left  to  right,  right  to 
left,  or  randomly.  In  addition,  speeds  of  1,  2,  or  3  m/sec  can  be  chosen. 

The  console  contains  the  programming  circuitry.  On  the  front  of  the 
console  are  the  various  switches  which  allow  the  user  to  select  from  the  large 
number  of  options  available.  In  addition,  a  CRT  monitor  depicts  the  relevant 
target  along  with  a  white  dot  indicating  point  of  aim.  After  a  round  is  fired, 
the  soldier  can  see  exactly  where  his  bullet  would  have  hit.  He  can  also  replay 
his  point  of  aim  for  the  1  sec  prior  to  firing.  In  addition  to  the  visual 
feedback,  the  user  can  also  receive  auditory  feedback  in  the  form  of  tone  pitch 
(low  tone-miss;  high  tone=hit)  and  a  voice  synthesizer  indicating  the  location 
of  misses  (e.g.,  "high-lef t") .  The  MTRMT  can  also  replay  point  of  aim  for 
moving  targets.  A  printer  is  provided  in  the  console  so  that  hard  copies  of 
performance  information  can  be  obtained.  For  a  more  detailed  description  of  the 
many  features  and  options  provided  by  the  MTRMT,  see  Spartanics,  Ltd.,  1967, 

1981  or  Schendel  and  Williams,  1982. 
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Figure  L.  Figure  l  shows  the  hardware  configuration  for  the  MTRMT 
simulator/trainer  for  the  M16A1  Rifle. 


The  second  simulator/ trainer  was  developed  by  the  present  author  at  the 
Army  Research  Institute,  Fort  Benning,  GA.  The  name  of  the  ARI  system  is  MACS 
(Multipurpose  Arcade  Combat  Simulator).  While  the  MTRMT  described  above 
represents  a  very  high  fidelity  and  very  realistic  simulator,  the  MACS  system 
represents  a  fairly  low  fidelity  part-task  simulator/ trainer  (e.g.,  there  is  no 
recoil  or  realistic  sound  built  into  the  MACS  system).  The  major  potential 
advantage  that  the  MACS  system  would  hold  over  the  MTRMT  would  be  the  cost 
involved:  the  components  for  the  MACS  system  are  much  simpler  and  less  costly 
and  the  MACS  system  has  potentially  more  training  applications  since  it  is  not 
dedicated  to  one  weapon. 

The  MACS  system  is  a  relatively  simple  system  composed  of  three  components 
(see  Figure  2).  The  heart  of  the  system  is  the  microcomputer  and  accompanying 
disk  drives.  Although  many  computers  could  potentially  be  used,  the  Apple  11+ 
was  used  in  the  MACS  prototype  system.  The  second  component  is  the  monitor. 
Virtually  any  NTSC  standard  monitor  should  work  with  the  system.  The  third 
component  is  the  ’'dummy"  weapon.  MACS  was  created  to  be  flexible  so  that  many 
different  weapons  could  be  trained/simulated  using  the  same  core  system.  The 
only  hardware  elements  that  would  change  from  weapon  to  weapon  are  the  "dummy" 
weapons  used  and  the  computer  software  that  controls  the  type  of  target  (e.g., 
personnel,  tank,  etc.)  or  nature  of  presentation  (e.g.,  stationary  or  moving 
target).  The  device  that  makes  the  MACS  system  possible  and  allows  for 
flexibility  is  the  light  pen.  The  light  pen  used  in  the  MACS  system  was 
specially  adapted  so  that  it  could  make  accurate  readings  out  to  5  m.  The  light 
pen  was  constructed  so  that  it  could  be  easily  attached  to  and  removed  from 
different  weapons. 

In  the  MACS  system,  the  user  can  select  from  several  training  options 
(e.g.,  stationary  targets,  moving  targets  with  or  without  replay,  effects  of 
wind,  etc.).  Scaled  targets  are  presented  on  the  monitor  representing  different 
distances  (virtually  any  reasonable  distance  can  be  displayed).  When  the  user 
fires  the  weapon,  the  computer  simulates  a  weapon  firing.  However,  this  sound 
is  merely  a  cue  that  the  weapon  has  fired  and  is  not  intended  to  simulate  the 
sound  intensity  of  actual  live  fire.  The  user  receives  both  visual  and  auditory 
feedback.  The  visual  feedback  is  in  the  form  of  a  dot  on  the  monitor  that 
symbolizes  where  the  bullet  would  have  impacted.  If  the  shot  is  relatively 
close  to  center  of  mass  (within  5  pixels  of  exact  center),  the  user  also 
receives  auditory  feedback  in  the  form  of  1  to  4  "beeps"  (the  more  beeps,  the 
closer  to  center  of  mass).  A  numerical  score  is  also  calculated,  presented,  and 
a  total  score  is  accumulated.  For  motivational  reasons,  the  MACS  system  was 
designed  to  resemble  an  arcade  game  as  much  as  possible.  For  more  detailed 
information  about  the  MACS  system,  see  Schroeder,  1983. 

The  purpose  of  the  present  study  was  to  measure  the  relationship  between 
the  two  simulators  for  both  stationary  and  moving  targets.  Of  course,  the  true 
test  of  any  simulator  is  the  magnitude  of  the  correlation  between  performance  on 
that  simulator  and  actual  performance  on  the  task  being  simulated.  Such  studies 
have  been  conducted  for  the  Weaponeer  (the  early  stationary-target  version  of 
the  MTRMT)  yielding  correlation  coefficients  of  .41  to  .66  reported  by  Schendel 
and  Heller  (1982)  and  .54  as  reported  by  Schendel,  Heller,  Finley,  and  Hawley 


(1983).  In  the  future,  additional  research  is  planned  to  determine  the 
relationship  of  both  of  these  simulators  to  live-fire  performance  and  attempts 
will  be  made  to  isolate  the  variables  that  significantly  influence  those 
relationships. 


Method 


Subjects . 


The  28  subjects  were  volunteers  from  the  staff  of  the  US  Army  Research 
Institute  for  the  Social  and  Behavioral  Sciences  (ARI),  volunteers  from  the 
staff  of  the  Mellonics  Systems  Development  Division,  Litton  Systems,  Inc.,  and 
volunteers  from  the  US  Army  Infantry  Center  at  Fort  Benning  GA.  Nine  of  the 
subjects  were  female  and  19  were  male.  The  marksmanship  experience  within  the 
group  ranged  from  novice  (never  had  held  a  rifle)  to  expert  (championship 
medalists) . 


Dependent  Measures. 


On  the  current  MACS  system,  the  computer  calculates  an  error  score  by 
adding  the  absolute  value  of  the  "XM  deviation  (the  difference  in  pixels  between 
the  X  coordinate  for  the  actual  center  of  mass  for  the  target  and  the  X 
coordinate  of  the  actual  shot  location)  and  the  absolute  value  of  the  "Y" 
deviation  (the  difference  in  pixels  between  the  Y  coordinate  for  the  actual 
center  of  mass  for  the  target  and  the  Y  coordinate  of  the  actual  shot  location) , 
and  subtracts  that  sum  from  50.  This  calculation  is  made  for  each  shot  and  the 
computer  adds  these  scores  to  determine  the  final  score. 

On  the  MTRMT,  no  numerical  score  is  provided  (other  than  hit/miss  data)  so 
a  numerical  score  representing  radial  error  had  to  be  created.  In  order  to 
derive  a  measurement  that  would  be  somewhat  comparable  to  the  MACS  measurement, 
a  plastic  template  was  constructed  with  22  concentric  circles  (the  radius  of 
each  circle  was  increased  by  3.175  mm).  This  template  was  fastened  over  the  CRT 
with  the  center  of  the  circles  positioned  directly  over  the  center  of  mass  on 
the  target.  The  center  circle  was  given  the  value  of  "0",  the  first  ring  from 
the  center  circle  was  given  the  value  ”1",  the  second  ring  "2",  etc.  For  each 
shot,  the  experimenter  recorded  the  value  of  the  ring  in  which  the  shot  hit. 
Because  the  targets  for  the  various  distances  are  not  all  presented  at  true 
scale  on  the  MTRMT,  trigonometric  corrections  were  made  on  these  scores  so  that 
corrected  scores  would  represent  radial  error  for  each  range.  Finally,  the 
corrected  scores  were  subtracted  from  50  as  in  the  MACS  system.  The 
manipulations  described  above  were  made  in  order  that  the  scoring  systems  for 
the  MACS  and  the  MTRMT  be  analagous  as  possible. 
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Procedure . 


Subjects  were  randomly  assigned  to  one  of  two  counterbalanced  conditions: 
MTRMT  followed  by  MACS  and  MACS  followed  by  MTRMT.  In  either  case,  testing 
conditions  were  kept  as  similar  as  possible  in  an  attempt  to  eliminate 
extraneous  confoundings .  On  both  simulators,  stationary  targets  were  presented 
first  followed  by  moving  targets.  For  stationary  targets,  subjects  received  the 
same  scenario  on  both  simulators  with  three  shots  allowed  at  each  target.  For 
both  simulators,  targets  were  presented  in  the  following  order:  100  m,  250  m, 

50  m,  150  m,  250  m,  150  m,  and  150  m.  Subjects  were  allowed  as  much  time  as 
they  wanted  to  take  the  three  shots.  For  both  simulators,  when  targets  were 
presented  at  the  same  range,  their  lateral  position  in  the  display  was  changed. 
During  the  moving  target  test,  targets  on  both  simulators  were  scaled  to 
represent  100m  targets  moving  at  2m/ sec.  Direction  of  movement  was  alternated 
with  targets  first  moving  from  left  to  right,  then  right  to  left,  etc.  A  total 
of  8  targets  were  presented  on  each  of  the  two  simulators.  All  subjects  were 
encouraged  to  utilize  the  feedback  provided  by  both  simulators.  Following  the 
data  collection,  all  subjects  were  given  a  short  debriefing  and  thanked  for 
their  participation. 

Results  and  Discussion. 


The  results  of  the  study  are  shown  in  Table  1.  As  shown,  all  correlations 
were  significant.  The  lowest  correlation  was  between  the  MACS  -  stationary 
targets  and  the  MACS  -  moving  targets  in  which  22%  of  the  variance  was 
explained.  The  two  strongest  correlations  found  were  the  MTRMT  -  moving  targets 
with  the  MTRMT  -  stationary  targets  in  which  56%  of  the  variance  was  explained, 
and  the  MTRMT  -  moving  targets  with  MACS  -  moving  targets  in  which  55%  of  the 
variance  was  explained.  Inspection  of  the  raw  data  indicated  that  there  were 
several  "outliers"  among  the  different  variables  which  may  have  artificially 
inflated  and/or  deflated  the  correlations  reported  in  Table  1.  For  the  purpose 
of  this  paper  an  outlier  was  defined  as  any  score  that  fell  more  than  2.33 
standard  deviations  away  from  the  mean  for  that  variable.  By  chance,  only  2%  of 
the  scores  would  be  expected  to  fall  beyond  2.33  standard  deviations  from  the 
mean.  In  order  to  assess  the  effects  of  these  outliers,  any  subjects  with  such 
deviant  scores  were  omitted  and  another  set  of  correlations  was  computed  (see 
Table  2).  Comparison  of  Table  1  and  Table  2  indicates  that  the  outliers  in  the 
present  data  did  inflate  most  of  the  original  correlations  (the  remaining  sample 
size  after  the  outliers  were  removed  is  shown  in  parentheses).  All  correlations 
decreased  in  magnitude  except  the  MACS  -  stationary  with  Weaponeer  -  stationary 
correlation,  which  remained  constant.  All  levels  of  significance  dropped, 
partly  because  of  smaller  correlation  coefficients  and  partly  because  of 
decreased  degrees  of  freedom. 
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In  general,  the  MACS  -  stationary  target  measure  had  the  lowest 
correlations  with  the  other  variables.  One  explanation  is  that  the  MACS  - 
stationary  target  scenario  was  too  easy  and  simply  didn't  allow  enough  variation 
in  performance.  Failure  to  allow  one  or  both  of  the  variables  enough  room  to 
vary  classically  results  in  a  low  correlation  coefficient.  This  explanation  is 
supported  by  the  fact  that  the  scores  on  the  MACS  -  stationary  target  measure 
were  more  highly  negatively  skewed,  indicating  the  possible  presence  of  a 
ceiling  effect.  The  coefficients  of  skew  for  the  four  measures  were:  MACS  - 
stationary  =  -1.77,  MACS  -  moving  =  -1.11,  MTRMT  -  stationary  =  -.52,  and  MTRMT 
-  moving  =  -.22. 

The  correlations  in  Table  1  are  relatively  high  for  the  general  small-arms 
simulation  arena.  This  is  probably  at  least  partly  due  to  the  fact  that  most 
correlations  reported  are  based  on  the  rather  crude  hit/miss  measure.  Using  a 
microcomputer-based  simulator  allows  radial  error  measures  to  be  calculated  and 
retained.  Being  a  more  precise  measure,  radial  error  has  the  effect  of  allowing 
the  dependent  variables  to  vary  more.  Hence,  the  chances  of  detecting  a 
relationship  are  increased. 

It  is  difficult  to  interpret  the  correlations  in  Tables  1  and  2  without 
some  measure  of  the  reliability  of  the  separate  devices  (i.e.,  the  extent  to 
which  each  device  correlates  with  itself).  This  information  is  not  yet 
available  for  MACS  and  cannot  be  estimated  from  the  present  results  because 
individual  score  data  were  not  retained  in  the  present  version  of  MACS. 

However,  individual  shot  data  were  recorded  for  the  MTRMT.  Reliability 
estimates  were  determined  for  the  150-m  stationary  target,  the  250-m  stationary 
target,  and  the  75-m  moving  target.  The  150-m  target  was  presented  a  total  of 
nine  times.  To  obtain  an  estimate  of  the  reliability  for  the  150-m  target,  the 
sum  of  the  scores  for  the  first,  third,  seventh,  and  ninth  presentations  was 
correlated  with  the  sum  of  the  scores  for  the  second,  fourth,  sixth,  and  eighth 
presentations.  The  score  for  the  fifth  presentation  was  not  used  in  order  to 
allow  an  equal  sample  size  for  both  sets.  The  reason  for  using  counterbalanced 
grouping  was  to  reduce  any  possible  systematic  effects  due  to  learning,  fatigue, 
etc.  The  resulting  correlation  for  the  150-m  target  was  r=».36,  p>.05.  However, 
when  outliers  were  removed  using  the  method  described  above,  the  correlation 
increased  to  r=.56,  p<.01.  The  250-m  target  was  presented  six  times.  The 
scores  for  these  six  shots  were  divided  into  two  subsets:  the  first,  third,  and 
sixth  scores  formed  set  one  and  the  second,  fourth  and  fifth  scores  formed  the 
second  set  for  odd-numbered  subjects;  and  the  first,  fourth,  and  sixth  scores 
formed  one  set  and  the  second,  third,  and  fifth  scores  formed  the  second  set  for 
the  even-numbered  subjects.  The  resulting  correlation  was  r=.75,  p<.001  for  the 
250-m  target.  Removing  outliers  using  the  method  described  above  slightly 
lowered  the  correlation  to  r=.72,  p<.001.  To  obtain  an  index  of  reliability  for 
the  75-m  moving  target  scenario,  the  first,  third,  sixth,  and  eighth  shots  were 
combined  to  form  one  set  and  the  second,  fourth,  fifth,  and  seventh  shots  were 
combined  to  form  the  second  set.  The  resulting  correlation  was  r=.50,  p<.01. 


Removal  of  the  outliers  resulted  in  a  nonsignificant  correlation  of  r=.35, 
p>.05.  Although  these  correlations  are  based  on  a  small  number  of  shots,  they 
should  provide  a  rou^h  estimate  of  the  reliability  inherent  in  the  MTRMT.  The 
correlations  found  between  Weaponeer  and  MACS  in  Tables  1  and  2  are  based  on 
more  scores  and  hence  are  more  stable  and  not  directly  comparable  with  the 
within-Weaponeer  reliability.  Nevertheless,  the  correlations  between  the  two 
simulators  do  seem  to  approach  the  within-simulator  reliability  of  one  of  the 
simulators . 

The  significant  positive  correlations  found  in  the  present  study  are 
encouraging.  It  appears  that  the  two  simulators  are  measuring  something  in 
common.  However,  more  research  is  needed  to  determine  the  extent  that  both 
these  simulators  are  correlated  with  actual  live  fire. 
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